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What Is The Nation's Report Card™? 

The Nation's Report Card™ informs the public about the academic achievement of elementary and secondary 
students in the United States. Report cards communicate the findings of the National Assessment of Educational 
Progress (NAEP), a continuing and nationally representative measure of achievement in various subjects over time. 

Since 1969, NAEP assessments have been conducted periodically in reading, mathematics, science, writing, U.S. 
history, civics, geography, and other subjects. NAEP collects and reports information on student performance at 
the national and state levels, making the assessment an integral part of our nation's evaluation of the condition and 
progress of education. Only academic achievement data and related background information are collected. The 
privacy of individual students and their families is protected. 

NAEP is a congressionally authorized project of the National Center for Education Statistics (NCES) within the Institute 
of Education Sciences of the U.S. Department of Education. The Commissioner of Education Statistics is responsible 
for carrying out the NAEP project. The National Assessment Governing Board oversees and sets policy for NAEP. 
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A New Generation 
of Testing 

Science education is not just about learning facts in a classroom — it's about doing 
activities where students put their understanding of science principles into action. 
That's why two unique types of activity-based tasks were administered as part of the 
2009 National Assessment of Educational Progress (NAEP) science assessment. In 
addition to the paper-and-pencil questions, fourth-, eighth-, and twelfth-graders also 
completed hands-on and interactive computer tasks. These tasks help us understand 
not only what students know, but how well they are able to reason through complex 
problems and apply science to real-life situations. While performing the interactive 
computer and hands-on tasks, students manipulate objects and perform actual 
experiments, offering us richer data on how students respond to scientific challenges. 

Here's what we learned about student performance across the tasks: 



Students were 
successful on parts 
of investigations that 
involved limited sets 
of data and making 
straightforward 
observations of 
that data. 



i 



Students were 
challenged by parts 
of investigations 
that contained more 
variables to manipulate 
or involved strategic 
decision making to 
collect appropriate 
data. ___ ____ 



The percentage 
of students who 
could select correct 
conclusions from 
an investigation 
was higher than for 
those students who 
could select correct 



conclusions 
and also 
explain their 
results. 



These three key discoveries will be discussed in more depth on pages 8 and 9. 
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EXPLORE THE TASKS 

This report is the print companion to the NAEP interactive website at http://nationsreportcard.gov/science_2009/ 



Science in Action 



Introduction 



Interactive computer and hands-on tasks were designed to assess how well students 
can perform scientific investigations, draw valid conclusions, and explain their results. 
As a part of the 2009 science assessment, a new generation of hands-on tasks was 
administered during which students worked with lab materials and other equipment 
to perform experiments. While hands-on tasks have been used in NAEP since the 
1990s, these new tasks present students with more open-ended scenarios that require 
a deeper level of planning, analysis, and synthesis. For the first time, the NAEP science 
assessment also included interactive computer tasks in science. 



The New Science Framework 

The National Assessment Governing Board oversees 
the development of NAEP frameworks that describe the 
specific knowledge and skills that should be assessed 
in each subject. Frameworks incorporate ideas and 
input from subject area experts, school administrators, 
policymakers, teachers, parents, and others. The 
2009 science framework was developed to keep the 
assessment content current with key developments in 
science standards, innovative assessment approaches, 
and recent research in both science and cognition. 

The 2009 science framework recommends that new 
types of performance-based tasks be assessed, including 
hands-on tasks and interactive computer tasks. These 
activity-based tasks allow us to examine students' 
abilities to combine their science knowledge with real- 
world investigative skills. 

Hands-on tasks are 40-minute activities where students 
use materials and laboratory equipment to perform 
actual science experiments. These tasks provide 
students an opportunity to demonstrate how well they 
are able to plan and conduct scientific investigations, 
reason through complex problems, and apply their 
scientific knowledge in real-world contexts. At each 
of the three grades, more than 2,000 students were 
administered the hands-on tasks. 

Interactive computer tasks are either 20 or 40 minutes 
in length and require students to solve scientific 
problems in a computer-based environment, often by 
simulating a natural or laboratory setting. These tasks 



provide students an opportunity to demonstrate a broad 
range of skills involved in doing science, but without 
many of the logistical constraints associated with 
the hands-on tasks. More than 2,000 students were 
administered the tasks at each of the three grades. The 
hands-on and interactive computer task assessments 
were given to separate national representative samples. 

The complete science framework for the 2009 
assessment is available at http://www.nagb.org/ 
publications/f rameworks7science-09.pdf 



Predict, Observe, Explain 

As suggested in the science framework, some 
of the hands-on tasks and interactive computer 
tasks use a Predict-Observe-Explain problem set 
to engage students in the scientific process. 

Predict 

Students provide a prediction for what might 
happen in a real-world science situation 

Observe 

Students conduct an investigation and observe 
what happens 

Explain 

Students explain what they have observed by 
interpreting data or drawing conclusions 
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Dig Deeper Online 




Take any of the nine 
interactive computer 
tasks and see how 
you would score 



http://nationsreporfcard.gov/science_2009/ 



Go behind 
the scenes 
of a grade 12 
hands-on task 






Students' prior 

knowledge 

assessed 




DETAILED RESULTS 




Students perform 
first sunlight 
investigation 

DETAILED RESULTS 


*J Students draw 
conclusions 


>— ► DETAILED RESULTS 



Only 23% of all fourth-graders displayed complex prior knowledge, did the experiment correctly, 
and were able to give complete explanations. (Follow the leftmost series of green disks) 



31% 

Simple 




10% 

Incorrect 



8% 
Correct 



23% 26% 2% 8% 8% 15% 1% 
Complete Partial or Complete Partial or Complete Partial or Complet 
Incorrect Incorrect Incorrect 



2% 

Incorrect 



7% 2% 6% <1% 2% 

Partial or Complete Partial or Complete Partial or 
Incorrect Incorrect Incorrect 
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A Look at the Tasks 



Hands-On Tasks 

The descriptions below provide an overview of the two hands-on tasks that were administered at each grade. 
To explore some of these tasks on the interactive website, visit the corresponding links. 



Grade 4 





How Seeds Travel 

In this five-part task, students investigate the structural characteristics 
of nine types of seeds to determine whether they are spread 
by wind or by animals. Students finish the task by predicting 
which seeds might travel the farthest and designing a possible 
investigation to test which seeds travel farther by wind. 

Electrical Circuits 

In this four-part task, students learn to assemble a simple 
electrical circuit. Then students investigate the conductivity 
of objects and the effect of multiple components in a 
series circuit. Finally, students use their knowledge 
learned from comparing different circuits to design 
and conduct an investigation to determine which 
of two black boxes contains a light bulb. 
http://nationsreportcard.gov/science_2009/hot_g4 
_scoring.asp 



Grade 8 





U 



Magnetic Fields 

In this three-part task, students design and conduct investigations 
based on observations of magnetic properties to determine what 
materials make up four metal bars. First, they use only the metal 
bars themselves. Students then repeat the investigation using 
a test magnet and compare the results of the investigations to 
confirm their conclusions. Finally, students design and conduct 
two different tests to compare the magnetic strength of a strong 
and a weak magnet. 

http://nationsreportcard.gov/science_2009/hot_g8_scoring.asp 
What's Cooking? 

In this two-part task, students investigate physical and chemical 
properties of four common cooking ingredients. Then students 
use the results of their first investigation to identify the ingredients 
in a mixture. 



_ 
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Plant Pigments 

In this two-part task, students investigate extracts from three unidentified 
organisms that were collected from the coastline of an island in the Pacific Ocean. 
Students determine what type of pigments each organism contains, and then they 
use their results together with other information to predict the type of organism 
that is most closely related to the unknown organisms. 

Maintaining Water Systems 

In this three-part task, students make a preliminary 
recommendation for which of two sites would be the 
better location for building a new town based on which 
site might have better water quality. Students then test 
water samples from both sites and determine whether the 
samples meet federal standards for various pollutants. 
Finally, students provide a final recommendation for the 
better site to build the town based on their results. 
http://nationsreportcard.gov/science_2009/hot_g12_scoring.asp 




Table 1. 

Average percent correct score for all hands-on tasks in NAEP science, by selected student characteristics and grade: 2009 

i Gender , , Race/ethnicity 1 r— Eligibility for NSLP 1 — i 





All 

students 


Male 


Female 


White 


Black 


Hispanic 


Asian/ 
Pacific 
Islander 


Eligible 


Not 
eligible 


Grade 4 


47 


45 


49 


51 


37 


42 


53 


41 


52 


Grade 8 


44 


43 


45 


48 


35 


37 


45 


38 


48 




40 


39 


41 


45 


29 


35 


43 







— Not available. 

National School Lunch Program. 

NOTE: Black includes African American, Hispanic includes Latino, and Pacific Islander includes Native Hawaiian. Race categories exclude Hispanic origin. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2009 Science Assessment. 



EXPLORE THE TASKS 

This report is the print companion to the NAEP interactive website at http://nationsreportcard.gov/science_2009/ 



Interactive Computer Tasks 

The descriptions below provide an overview of the three interactive computer tasks that were administered at each grade. 
To explore each of these tasks on the interactive website, visit http://nationsreportcard.gov/science_2009/ict_tasks.asp. 



Grade 4 




Lid 




Mystery Plants 

In this 40-minute extended task, students use a simulated greenhouse to determine the best 
sunlight or fertilizer amounts for two different plants. Students begin the task by showing 
their prior knowledge about how sunlight and nutrients are related to optimal plant 
growth. Then students run three separate investigations and draw their 
conclusions about the effect of sunlight and nutrients on the plant samples. 



Cracking Concrete 

In this 20-minute task, students investigate what happens to the volume of water 
when it freezes. Then students use the results of their investigations to predict 
and test what will happen when water freezes in the cracks of a concrete sidewall< 



Here Comes the Sun 

In this 20-minute task, students use a time 
lapse simulation to make observations about 
the path of the sun as it relates to the amount 
of daylight. Students use this knowledge to 
determine the better of two locations for 
growing tomatoes. 



Grade 8 





V J 




Bottling Honey 

In this 20-minute task, students investigate how four different liquids behave 
when they are poured and how temperature affects the flow rates of the 
liquids. Then students determine the best temperature range for bottling honey 
that will take the least amount of time while using as little energy as possible. 

Playground Soil 

In this 20-minute task, students investigate the permeability of soil samples 
from two sites a town is considering for a play area. Students use their results to 
help decide which site has the better water drainage and is therefore the better 
place for a grassy play area. 

Planning a Park 

In this 40-minute extended task, students help plan a new recreation area for 
a town using a small portion of an existing wildlife area. Students evaluate the 
potential impact that various locations of the recreation area would have on 
the population of the meadow vole and other animals. By the end of the task, 
students make a recommendation for the best placement of the new park. 
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All of these tasks have been released to the public and can 
be used by parents, students, and teachers by visiting 
http://nationsreportcard.gov/science_2009/ict_tasks.asp. 



Grade 12 






Starlight 

In this 20-minute task, students look at a way of classifying stars based 
on their temperatures and luminosities. Students then compare data on 
two stars, and predict how characteristics like their temperature and 
luminosity might change throughout their lives. 

Energy Transfer 

In this 20-minute task, students investigate which metal would be 
better for making the bottom of a cooking pan. While designing and 
conducting their investigations, students use a simulated calorimeter to 
test the specific heat capacities of two metals that could be used for the 
bottom of the pan. 

The Phytoplankton Factor 

In this 40-minute extended task, students investigate the role of 
phytoplankton (microscopic, plant-like organisms that live near the 
ocean surface) in the Earth's carbon cycle. In addition, students 
analyze an authentic set of experimental data relating levels of iron 
and nutrients to the growth of phytoplankton, and use a resource 
library to research ocean locations where increased iron levels might 
affect phytoplankton growth. 




Table 2. 

Average percent correct score for all interactive computer tasks in NAEP science, by selected student characteristics 
and grade: 2009 

i Gender , , Race/ethnicity 1 r— Eligibility for NSLP 1 





All 

students 


Male 


Female 


White 


Black 


Hispanic 


Asian/ 
Pacific 
Islander 


Eligible 


Not 
eligible 


Grade 4 


42 


42 


42 


47 


32 


36 


49 


36 


48 


Grade 8 


41 


40 


41 


45 


34 


33 


50 


34 


46 




27 


27 


28 


30 


19 


24 


33 







— Not available. 

1 National School Lunch Program. 

NOTE: Black includes African American, Hispanic includes Latino, and Pacific Islander includes Native Hawaiian. Race categories exclude Hispanic origin. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2009 Science Assessment. 



EXPLORE THE TASKS 

This report is the print companion to the NAEP interactive website at http://nationsreportcard.gov/science_2009/ 
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What Did We Learn? 



Key Discovery 1 



Key Discovery 2 



Students were successful on 
parts of investigations that 
involved limited sets of data 
and making straightforward 
observations of that data. 



Students were challenged 
by parts of investigations 
that contained more variables 

to manipulate or involved 
strategic decision making to 
collect appropriate data. 



FOR EXAMPLE 



FOR EXAMPLE 



Mystery 
Plants 



Playground 
Soil 



Maintaining 
Water 
Systems 



GRADE 4 

80% of students could use a simulated 
greenhouse to test how three levels 
of sunlight affected plant growth. 
Students could use six different trays 
of the same plant type to test the three 
conditions, which allowed for a more 
straightforward observation. 

GRADE 8 

84% of students could use a simulated 
laboratory to test how much water 
flowed through two different soil 
samples. Students who did this correctly 
made the straightforward observation 
that one soil sample allowed more water 
flow than the other. 

GRADE 12 

75% of students could use test 
strips to test water samples for the 
levels of four pollutants, record the 
data, and interpret whether the results 
exceeded EPA standards. This part of 
the investigation was straightforward 
because students did not have to 
manipulate variables. 




Mystery 
Plants 




Magnetic 
Fields 




Energy 
Transfer 



GRADE 4 

35% of students could select from nine 
possible fertilizer levels to test and determine 
those best for growth of a sun-loving plant. 
However, students had only six trays available 
during any single test run; therefore, students 
had to make strategic choices to assure that 
an adequate range of data was sampled. 

GRADE 8 

24% of students could appropriately decide 
how to manipulate four metal bars made of 
unknown materials to determine which ones 
were the magnets. Because students could 
use only the four bars for this investigation, 
they had to apply their knowledge of how to 
test for magnetic properties. 

GRADE 12 

25% of students designed and conducted 
an investigation using a simulated calorimeter, 
and related patterns in temperature changes 
in two different metals to decide which metal 
has the higher specific heat capacity. Students 
had to interpret this complex set of data and 
relate it to their knowledge that the metal 
with the higher specific heat capacity caused 
the temperature of the water to change more 
than the metal with the lower specific 
heat capacity. 
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Key Discovery 3 

The percentage of students 
who could select correct 
conclusions from an 
investigation was higher 
than for those students 
who could select correct 
conclusions and also 
explain their results. 

FOR EXAMPLE 



Table 3 provides a summary of the results used to 
support the three key discoveries from the hands-on and 
interactive computer tasks. The mean percent correct 
represents the proportion of students who answered 
multiple-choice questions correctly and accounts for at 
least partial credit on constructed-response questions. 
Science content experts examined all of the scored tasks 
to find the patterns in the results. The table shows the 
minimum, maximum, and median percent correct to 
demonstrate the ranges of student performances on the 
skills identified within each of the key discoveries. 

Table 3. 

Percent correct supporting key discoveries for hands-on and 
interactive computer tasks in NAEP science at grades 4, 8, 
and 12: 2009 



Number of items Percent correct 

across grades and 
assessment type Minimum Maximum Median 



Cracking 
Concrete 



Key Discovery 1: 
Straightforward observations 


12 


42 


94 


76 


Key Discovery 2: 
Strategic decision making 


11 


22 


63 


36 


Key Discovery 3: 
Correct conclusion 


9 


35 


93 


71 


Key Discovery 3: 
Explain results 


9 


9 


44 


30 



Bottling 
Honey 



GRADE 4 

71% of students could select the 
correct conclusion about how volume 
changes when ice melted into water, 
while 15% could select the correct 
conclusion and support this conclusion 
with evidence from the investigation. 

GRADE 8 

88% of students could select the liquid 
that flowed at the same rate as water at 
a given temperature, while 54% could 
select the correct liquid and support 
this conclusion in writing using evidence 
from their investigation. 

GRADE 12 

55% of students were able to select 
the correct temperature changes that 
occurred when a warm solid object was 
placed into cool water, while 27% were 
able to make the correct selections and 
explain how heat was transferred from a 
warmer to a cooler substance. 

EXPLORE THE TASKS 

This report is the print companion to the NAEP interactive website at http://nationsreportcard.gov/science_2009/ 




Energy 
Transfer 



SOURCE: U.S. Department of Education, Institute of Education Sciences, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2009 Science Assessment. 



Both the hands-on and interactive 
computer tasks can be successfully 

administered in a large-scale 
assessment setting, usin 
standardized, controlled 
procedures so as to ensure 
the collection of valid 
assessment data. 
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What Else Did We Learn? 



Results for the interactive computer and hands-on tasks can also be reported by 
race/ethnicity, gender, and eligibility for the National School Lunch Program. Many of 
the results for groups are consistent with the findings from the main paper-and-pencil 
NAEP science test; however, there were some differences. The full results by student 
group are available in tables 1 and 2, and the main paper-and-pencil science results are 
available online at http://nationsreportcard.gov/science_2009. 



Female students in all three grades 
scored higher than males on the 
hands-on tasks, though males scored 
higher on the traditional paper-and- 
pencil science assessment. There 
was no gender gap in interactive 
computer tasks. 

At grades 4 and 12, Hispanic students 
scored higher than their Black peers on 
both the hands-on tasks and interactive 
computer tasks. 

White and Asian/Pacific Islander 
students in all three grades scored 
higher than their Black and Hispanic 
peers on both the hands-on tasks and 
interactive computer tasks. 



There was no score gap between 
White and Asian/Pacific Islander 
students in any of the three grades on 
the interactive computer and hands-on 
tasks; however, on the main science 
assessment, White students scored 
higher at grades 4 and 8. 

There was an achievement gap at 

grades 4 and 8 between students from 
higher- and lower-income families in 
both the hands-on tasks and the 
interactive computer tasks. 




10 



The Nation's Report Card 



What Are Students Doing 
in Science Classrooms? 



As part of the main paper-and-pencil 2009 science assessment, students and 
teachers answered survey questions about science learning and instruction. These 
examples provide some context for student performance on the interactive computer 
and hands-on tasks in science. The full results for these contextual variables are in tables 
A-D on page 19. 



■ 39% of fourth-graders and 57% 

of eighth-graders had teachers who 
reported at least a moderate emphasis 
on developing scientific writing skills. 

■ 28% of twelfth-graders reported 
writing a report on a science project 
at least once a week. 



51% of twelfth-graders reported 
designing a science experiment at 
least once every few weeks. 

53% of all twelfth-graders reported 
that they were currently taking a 
science course. 

92% of fourth-graders and 98% 
of eighth-graders had teachers who 
reported doing hands-on activities 
with students at least monthly. 
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nside the Tasks 



Hands-On Task— Maintaining Water Systems 



For this task, grade 12 students were asked to investigate the best site for building a 
new town based on the quality of a given water supply. Using the provided laboratory 
equipment and materials, students had to test water samples for levels of specific 
pollutants and evaluate water treatment processes. 



Below are the results for each step of the experiment students performed. 



Step 1: Predict 

Students made a preliminary 
recommendation for the site of a 
new town based on the information 
provided about the quality of 
water sources. 

64% of students 
explained their preliminary 
recommendations with 
valid support based on the 
materials in their kits. 



Step 2: Observe 

Students performed water tests and 
evaluated data in comparison to 
national drinking water standards. 



75% of students could 
perform a straightforward 
investigation to test the 
water samples and 
accurately tabulate data. 



Step 3: Explain 

Students made a final 
recommendation for the site of a 
new town based on all of their data. 
Regardless of their performance on 
the first two steps, twelfth-graders 
struggled to explain their results. 

11% of students were 
able to provide a valid 
final recommendation 
by supporting their 
conclusions with details 
from the data. 



Steps 4 and 5: Extend 

Students extended their inquiries by matching pollutants to specific 
water treatment steps and describing these processes in detail. 

14% of students were able to correctly evaluate water treatment steps and select 
those that would be needed to remove pollutants that exceed national drinking 
water standards. 

28% of students were able to describe scientific processes used to remove 
water pollutants. 
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Table 4. 

Percentage of twelfth-grade students who successfully completed each step of the Maintaining 
Water Systems hands-on task in NAEP science, by selected student characteristics: 2009 



_ Gender. 



Race/ethnicity _ 





Male 


Female 


White 


Black 


Hispanic 


Asian/ 
Pacific 
Islander 




63 


64 


68 


39 


58 


76 


Step 2 


74 


76 


84 


50 


59 


66 




9 


13 


13 


3 


6 


17 


Step 4 


17 


11 


17 


1 


6 


13 


Step 5 


30 


27 


31 


13 


19 


39 



_ 

> 
_ 

in 
_Q 

O 

CL 



c 

CD 



NOTE: Black includes African American, Hispanic includes Latino, and Pacific Islander includes Native Hawaiian. Race categories exclude 
Hispanic origin. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP), 2009 Science Assessment. 



Watch a short online video to learn more about 
hands-on tasks and to view a demonstration 
of twelfth-grade students completing the 
Maintaining Water Systems experiment. 



EXPLORE THE TASKS 

This report is the print companion to the NAEP interactive website at http://nationsreportcard.gov/science_2009/ 
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Interactive Computer Task— Mystery Plants 



Grade 4 students designed and conducted three different experiments in this 
task, with the difficulty increasing as they proceeded. They were given a series of 
simulations and asked to determine the following: 



Experiment 1: What are the best sunlight conditions for growth for Plant A (a sun-loving plant)? 
Experiment 2: What are the best sunlight conditions for growth for Plant B (a shade-tolerant plant)? 
Experiment 3: What are the best fertilizer amounts for growth for Plant A? 



All of the experiments above required students to make predictions and observations and to explain their conclusions. 
Below are the results for each step of the three experiments that students performed. 



Step 1: Predict 

Students were asked about the 
sunlight and fertilizer that plants 
need. They were assessed on their 
ability to understand that different 
plants need different amounts 
of each. 

59% of students displayed 
complex prior knowledge 
in experiments 1 and 2, 
understanding that 
different plants have 
different sunlight needs. 

56% of students displayed 
complex prior knowledge 
in experiment 3, 
understanding that 
different plants have 
different fertilizer needs. 



Step 2: Observe 

Students were asked to observe 
and test across the range of 
available sunlight and fertilizer 
amounts, and investigate how these 
amounts correspond to the growth 
of the plants. 

At least 80% of students 
correctly performed this 
step in experiments 1 and 2, 
which involved limited sets 
of data and straightforward 
observations. 

35% of students could 
correctly perform 
experiment 3, which 
contained more variables 
and required them to 
make strategic decisions 
about the best fertilizer 
levels for growth of a 
sun-loving plant. 



Step 3: Explain 

Students were required to select 
the correct conclusion for each 
investigation and provide an 
explanation for each. 

While a higher percentage 
of students could select the 
correct conclusion for each 
of the three experiments, 
36% of students could 
explain their conclusions 
with supporting evidence 
from their investigation in 
experiment 1, 29% in 
experiment 2, and 46% 
in experiment 3. 
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Table 5. 

Percentage of fourth-grade students who successfully completed each step of the Mystery Plants (Experiment 1: Sunlight for 
Plant A) interactive computer task in NAEP science, by selected student characteristics: 2009 



Gender , , Race/ethnicity , , — Eligibility for NSLP 1 







Male 


Female 


White 


Black 


Hispanic 


Asian/ 
Pacific 
Islander 


Eligible 


Not 
eligible 


edict 




59 


58 


65 


49 


47 


67 


52 


63 






















a; 
> 




















_ 

LO 
_Q 


Step 2 


80 


79 


81 


79 


74 


86 


78 


81 


o 








































Expla 




34 


37 


43 


19 


27 


47 


28 


43 



1 National School Lunch Program. 

NOTE: Black includes African American, Hispanic includes Latino, and Pacific Islander includes Native Hawaiian. Race categories exclude Hispanic origin. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2009 Science Assessment. 



Try it out! Go online to take 
Mystery Plants, or any of the 
other interactive computer tasks, 
and see how you would score. 

http://nationsreportcard.gov/ 

science2009ict/mysteryplants/ 

mysteryplants.aspx 
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This report is the print companion to the NAEP interactive website at http://nationsreportcard.gov/science_2009/ 
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How Did Students Perform? 

A step-by-step look at Experiment 1 of Mystery Plants 



59% 

Complex 



31% 

Simple 



10% 

Incorrect 





What prior knowledge did students bring to the problem? 

Over one-half (59%) of all fourth- 
graders began the task with the 
understanding that different plants 
have different sunlight needs. 



Approximately 31% thought that 
all plants needed more sunlight 
to grow best. 



About 10% thought that sunlight 
didn't affect plants or thought 
that plants needed only a little 
bit of sunlight. 




o 




23% 

Correct 



8% 

Incorrect 



8% 

Correct 



2% 

Incorrect 



How well did students perform when using the simulated greenhouse? 

Regardless of the prediction made based on their prior knowledge, fourth-graders were able to perform this 
straightforward observation. Almost half (49%) of students showed complex prior knowledge and performed 
the experiment correctly, and about 80% of all students were able to perform this step correctly. 



j i 



1 i 



23% 26% 2% 8% 

E Complete Partial Complete Partial 
15 or Incorrect or Incorrect 



8% 15% 

Complete Partial 

or Incorrect 



1% 7% 

Complete Partial 

or Incorrect 



2% 6% <1% 2% 

Complete Partial Complete Partial 

or Incorrect or Incorrect 



How well could students explain their conclusion? 

23% of fourth-graders displayed complex prior knowledge, conducted the experiment correctly, and were able 
to explain the findings (as seen by following the leftmost series of green disks). However, the bottom row of disks 
shows that 36% of students were able to provide a complete explanation, so some students who did not have 
complex prior knowledge or did not complete the experiment correctly were able to give complete explanations. 
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Technical Notes 



Assessment Design 

Science Interactive Computer Tasks (ICTs) and 
Hands-On Tasks (HOTs) were administered in 2009 
at grades 4, 8, and 12. The two assessments were 
given to separate nationally representative samples; 
therefore, the results are not linked to each other 
or to the main operational science assessment. 
See http://nationsreportcard.gov/science_2009/ 
ict_indepth.asp for more details on example tasks in 
the assessments. 

Sampling and Accommodations 

The target population for the ICT and HOT assessments 
consisted of fourth-, eighth-, and twelfth-graders 
enrolled in public and private schools nationwide. 
Each school that participated in the assessment, and 
each student assessed, represents a portion of the 
population of interest. Results are weighted to make 
appropriate inferences between the student samples 
and the respective populations from which they are 
drawn. While part of the sample, there were insufficient 
American Indian/Alaska Native students assessed to 
permit reporting. In addition, participation rates fell 
below the 70 percent guideline for private schools, and 
therefore results cannot be reported separately. 

The results for the ICT and HOT assessments are 
based on administration procedures that allowed 
accommodations for students with disabilities (SD) and 
English language learners (ELL) selected to participate 
in the two assessments. Appropriate accommodations 
were determined by school officials. Read-aloud 
accommodations were provided for HOTs and short 
ICTs, but were not provided for the extended ICTs. As a 
result, a small portion of students in the ICT assessment 
who required read-aloud accommodations were only 
given the two short ICTs at that grade level. See tables 
showing the accommodations and participation rates at 
http://nationsreportcard.gov/science_2009/ict_tech_ 
notes. asp. 

National School Lunch Program 

NAEP collects data on student eligibility for the National 
School Lunch Program (NSLP) as an indicator of low 
income. Under the guidelines of NSLP, children from 
families with incomes below 130 percent of the poverty 
level are eligible for free meals. Those from families with 
incomes between 130 and 185 percent of the poverty 
level are eligible for reduced-price meals. (For the period 
July 1, 2008, through June 30, 2009, for a family of 
four, 130 percent of the poverty level was $27,560, and 



185 percent was $39,220.) Some schools provide free 
meals to all students irrespective of individual eligibility, 
using their own funds to cover the costs of noneligible 
students. Under special provisions of the National 
School Lunch Act intended to reduce the administrative 
burden of determining student eligibility every year, 
schools can be reimbursed based on eligibility data for 
a single base year. Participating schools might have 
high percentages of eligible students and report all 
students as eligible for free lunch. Because students' 
eligibility for free or reduced-price school lunch may be 
underreported at grade 12, the results are not included 
in this report but are available on the website at http:// 
nationsreportcard.gov/science_2009/. 

Reporting Results 

As with all other NAEP assessments, student responses 
to constructed-response items were scored according 
to standard scoring procedures. The data from scoring 
were then analyzed to create summaries of student 
performance, as shown in this report. As shown in 
the summary of major results and the examples for 
ICTs and HOTs, an item percentage correct statistic was 
used to summarize student performance. This statistic 
represents the percentage of examinees who received 
a correct score on the question for a multiple-choice 
or dichotomous constructed-response question. For 
a multilevel constructed-response question, the item 
percentage correct statistic is calculated by summing a 
weighted percentage of students attaining each score 
level. The weight is based on the number of levels in the 
scoring criteria for the question. 

Student performance across the three ICTs or two 
HOTs per grade level was also summarized as a student 
percent correct score. This percentage was calculated 
as the total score for a student across multiple tasks 
in the assessment and then divided by the maximum 
possible score for the questions the student attempted 
and multiplied by 100. For example, suppose a student 
attempted five questions in the first ICT task, four in 
the second task, and four in the third task, yielding a 
total score of 30. (Note that constructed-response 
items are "weighted" based on the number of score 
categories, e.g., a 4-category item has a weight of 3 with 
students getting 0, 1, 2, or 3 points credit on the item.) 
In addition, suppose that the maximum possible score 
for the 13 items the student attempted is 45. Then the 
student's percent correct score would be 30 divided 
by 45 multiplied by 100, which equals 67. The sum of 
scores for those items that students attempted, not all 
the items that appeared in an assessment, is used as 
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the denominator of the student percent correct score. 
This method is used because NAEP assessments are 
intended to be non-speeded, implying that students 
should not be penalized for failing to reach particular 
questions because of time limitations. More information 
on the percent correct scores is available at http:// 
nationsreportcard.gov/science_2009/ 
ict_tech_notes.asp. 



Interpreting the Results 

NAEP reports results using widely accepted statistical 
standards; findings are reported on a statistical 
significance level set at .05 with appropriate 
adjustments for multiple comparisons. Only those 
differences that are found to be statistically significant 
are discussed as higher or lower. 



Table A. 

Percentage of students in NAEP science, by teachers' responses to a question about emphasizing the development of scientific 
writing skills when teaching science and grade: 2009 



To what extent do you emphasize developing scientific 
writing skills in teaching science to your class? 


Not at all 


Small extent 


Moderate extent 


Large extent 


Grade 4 


14 


47 


31 


8 


Grade 8 


4 


38 


41 


16 



NOTE: Detail may not sum to totals because of rounding. 



Table B. 

Percentage of students in NAEP science, by teachers' responses to a question about students doing hands-on activities or 
investigations in science class and grade: 2009 



About how often do your science students do 
hands-on activities or investigations in science? 


Never or 
hardly ever 


Once or 
twice a month 


Once or 
twice a week 


Every day or 
almost every day 


Grade 4 


8 


42 


40 


11 


Grade 8 


2 


25 


56 


17 



NOTE: Detail may not sum to totals because of rounding. 



Table C. 

Percentage of students in twelfth-grade NAEP science, by their responses to a question about doing various activities during the 
year in science class: 2009 



In your science class this year, how often 
do you do the following activities? 


Never or 
hardly ever 


Once every 
few weeks 


About once 
a week 


Two to three 
times a week 


Every day or 
almost every day 


Write a report on your science project 
or activity 


39 


33 


20 


6 


2 


Design a science experiment 


49 


28 


16 


5 


2 



NOTE: Detail may not sum to totals because of rounding. 



Table D. 

Percentage of students in twelfth-grade NAEP science, by their responses to a question about taking a science course: 2009 



Are you currently taking a science course? 


Percentage of students 


Yes 


53 


No 


47 



NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2009 
Science Assessment. 
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